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What students think and what they actually do in a 
mobile assisted language learning context: new insights 
for self-directed language learning in higher education 

Gustavo Garcia Botero 1 and Frederik Questier 2 



Abstract. In an attempt to understand whether Mobile-Assisted Language 
Learning (MALL) could foster students’ self-directed learning, this paper analyzes 
a self-directed learning experience by means of a language app: Duolingo. In 
this study, higher education language students were encouraged to use Duolingo 
outside of the classroom. The data collected via app tracking, surveys and semi- 
structured interviews reveal that the low activity in the app contrasts the high 
value students attribute to it. Students indicated that the low activity is due to other 
obligations in their lives. They also expressed the need of external motivation to 
finish the course. The study suggests that mentoring and modeling are still needed 
in the development of self-directed study skills and it highlights the importance 
of implementing different data collection techniques to understand what students 
think and do in MALL. 
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1. introduction 

When inquiring about what students do in a computer assisted language learning 
environment, Stockwell (2012) hints at an over reliance on surveys as a data 
collection method. Likewise, Fisher (2012) states that the “use of questionnaires 
alone is not a reliable source of information about student use of software and 
should be avoided as a single source of information” (p. 27). The ubiquity and 
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just in time learning provided by MALL have particular implications on how 
users self-direct their learning and how researchers collect data on students’ 
activity. Stockwell (2013) points out the small “amount of research examining 
how learners engage in mobile learning outside of the classroom” (p. 118). Little 
is known about what happens when students download a language application to 
their devices. 

Conveniently, MALL applications are gradually adding features that allow 
quantitative measurements of app usage. One example is the free of charge 
application Duolingo. Through its unobtrusive tracking dashboard Duolingo for 
schools, teachers can see the languages students are practicing and their progress 
in the Duolingo language curriculum. 


2. Method 

The participants were Colombian higher education students enrolled at different 
levels of language courses. 574 students were given a formal lecture about 
Duolingo. They learned about its different features and they were encouraged 
to use the app at their convenience. Furthermore, students were invited to join 
the Duolingo dashboard where their activity would be tracked. 273 students 
reported to not have used Duolingo; 149 students reported having used Duolingo 
after the MALL lecture while 118 of these users accepted to be tracked in the 
dashboard. 

Eight weeks after the lecture in which they were encouraged to use the app, students 
filled out a questionnaire on their perception of Duolingo. The questionnaire 
contained items measured through a Likert scale where 1 represented totally 
disagree and 5 represented totally agree. 

Once a year had passed, students’ data from the dashboard were retrieved. A 
stratified sampling was applied to select students for an interview about their 
general impressions of the app and their opinion about how to foster its use 
among the student population. The analysis of the sampling population resulted 
in the stratification of the data into three groups of students: tracked students who 
barely used Duolingo (students who were active ten days or less, N=73), students 
who had a moderate use (students who were active more than ten and less than 
50 days, N=31) and students who had the most sustained use (students that were 
active 50 or more days, N=14). Five students in each group were randomly 
selected for the interview. 
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3. Results 


Table 1. Questionnaire: students’ perceptions about Duolingo 


Duolingo item sorted from high to low score. 

N=149 

Mean 

SD 

1. Duolingo encourages me to learn different languages 
that are not taught in the Modem Languages program 

3.97 

0.993 

2. 1 would like Duolingo to be integrated as a component of 
independent learning (work done outside the classroom) 

3.94 

0.91 

3. 1 like the game-like methodology proposed by Duolingo 

3.93 

0.844 

4. Duolingo allows me to be independent in language learning 

3.93 

0.844 

5. 1 have no problem to access Duolingo 

3.93 

1.101 

6. 1 would recommend Duolingo for language learning 

3.91 

0.932 

7. 1 find a connection between what I studied in Duolingo 
and what was taught to me by the language teachers 

3.89 

0.927 

8. 1 think I can improve my vocabulary with Duolingo 

3.89 

0.934 

9. 1 think Duolingo offers an organized language learning path 

3.87 

0.935 

10. Duolingo is fun for people my age 

3.87 

0.903 

1 1 . 1 like the exercises proposed by Duolingo 

3.86 

0.923 

12. Duolingo motivates me to study languages 
with my Smartphone or tablet 

3.85 

0.964 

13.1 think I can improve my grammar with Duolingo 

3.83 

0.918 

14. 1 think that I can improve my listening with Duolingo 

3.81 

0.896 

15.1 think I can improve my writing with Duolingo 

3.79 

0.903 

16. 1 think that I can improve my speaking with Duolingo 

3.72 

0.958 

17. 1 think Duolingo can satisfy my learning needs 

3.72 

0.985 

18. 1 would like my teacher to see my progress in Duolingo 

3.71 

1.009 

19. Duolingo encourages me to go beyond what 
is taught in the language classroom 

3.70 

0.991 

20. 1 think I can improve my reading with Duolingo 

3.68 

0.939 

21.1 would like Duolingo to be integrated inside the classroom 

3.58 

0.952 

22. 1 prefer other free apps to learn languages than Duolingo 

3.38 

0.977 


Table 2. Duolingo dashboard: average of completed units and number of days 
of Duolingo use 


N= 118 

Mode 

Range 

Mean 

SD 

SE 

Days Active 

1 

0- 394 

15.13 

45.18 

3.06 

Lessons completed 

0 

1908 

56.49 

169.95 

11.53 

Units completed* 

0 

78 

13.55 

18.47 

1.25 


* A unit is a set of lessons 
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Table 3. Course completion: verall language course completion in Duolingo 


Course Group 

N. 

N. Students 
who finished 
the course 

Language completed 

English semester 1 

17 

1 

English 

English semester 2 

18 

1 

English 

English semester 3 

13 

3 

English 

English semester 4 

20 

2 

English 

French semester 1 

8 

0 

- 

French semester 2 

10 

0 

- 

French semester 3 

14 

2 

1 English 

1 French 

French semester 4 

18 

3 

3 French 

1 Italian* 

Total 

118 

12 

8 English 

4 French 

1 Italian* 


*Same student did French and Italian 


4. Discussion 

According to the students’ questionnaire answers, Duolingo is perceived to 
be a useful tool for language learning. Views of Duolingo are positive as an 
instructional method (items 3, 7, 9, 11), motivation tool (items 1, 10, 11), 
promoter of self-directed learning (items 2, 4) and enabler of language skills 
practice items (8, 13-16, 20). These results were confirmed by the individual 
semi-structured interviews in which every interviewed student reported 
having recommended the app to peers and friends (as predicted in item 6 of 
the questionnaire). However, the results retrieved from the Duolingo dashboard 
reveal that only 12 out of the 118 tracked students (10%) managed to completely 
finish a Duolingo language course. Despite the considerable variability of the 
data, Table 2 puts into evidence the low participation of students. Different data 
collection techniques reveal a discrepancy between students’ perception and 
actual use of the application. Such discrepancy is seen in similar studies (e.g. 
Stockwell, 2008). From the students’ point of view, this is explained by their 
busy lives inside and outside their academic context, the lack of formal follow- 
up (related to the absence of a teacher), and the lack of additional motivational 
factors (e.g. language certificate, course grades). Our results confirm studies that 
take into account the relationship between students and technology (e.g. Azevedo 
et ah, 2003) in which the difficulty of learners to self-regulate when learning is 
demonstrated. 
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5. Conclusion 

This study points out the relevance of data collection to improve the understanding 
of MALL outside formal educational contexts. Although tracking is not the panacea 
in computer-assisted language learning research to investigate student behavior 
(Fischer, 2007), it demonstrates the relevance of scrutinizing actual usage data 
because what learners indicate in surveys contradicts what they do in reality. 

The combination of data collection techniques suggests that self-directed 
learning in MALL requires modeling and mentoring, which is why a tutor (such 
as a teacher) remains important. Even if students are formally introduced to an 
expectedly convenient and motivating app, only a small number of them make 
substantial extracurricular use of it. Residts of the study show that for substantial 
MALL usage, a combination of incentives, scaffolding and curricular integration 
are needed. Such concepts will be investigated in follow up studies. 
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